Associating comments with playback of media content

ABSTRACT

Methods, systems, and devices are described for associating comments with playback of media content. In these methods, systems and devices, a user input device receives a selection of a first point during the playback of the media content for insertion of a first comment. The input device may associate a first time code with the selected first point. The input device may also receive a comment from the user at a second point during the playback of the media content, the second point being different from the first point. The comment is associated with the first time code.

CROSS REFERENCES

The present application claims priority to the following patentapplications: U.S. Provisional Patent Application No. 61/421,945, filedDec. 10, 2010, entitled “DYNAMIC MULTIMEDIA COMMENT AND DISPLAYARCHITECTURE”; U.S. Provisional Patent Application No. 61/432,395, filedJan. 13, 2011, entitled “SYSTEM, DEVICE, AND INTERFACE ARCHITECTURE FORCOMMENT CREATION”; U.S. Provisional Patent Application No. 61/432,397,filed Jan. 13, 2011, entitled “SCREEN LOCATION COMMENT ARCHITECTURE”;U.S. Provisional Patent Application No. 61/439,189, filed Feb. 3, 2011,entitled “USER SELECTABLE COMMENT FILTER AND RELATED ARCHITECTURE”; andU.S. Provisional Patent Application No. 61/439,196, filed Feb. 3, 2011,entitled “COMMENT DELIVERY ARCHITECTURE,” each of which is incorporatedby reference in its entirety for all purposes.

The present application is further related to U.S. patent application,filed concurrently herewith, entitled “RECOGNITION LOOKUPS FORSYNCHRONIZATION OF MEDIA PLAYBACK WITH COMMENT CREATION AND DELIVERY”(Attorney Docket No. P003.02 (77828.0010)); U.S. patent application,filed concurrently herewith, entitled “PARALLEL ECHO VERSION OF MEDIACONTENT FOR COMMENT CREATION AND DELIVERY' (Attorney Docket No. P003.03(77828.0011)); U.S. patent application, filed concurrently herewith,entitled “ASSOCIATION OF COMMENTS WITH SCREEN LOCATIONS DURING MEDIACONTENT PLAYBACK” (Attorney Docket No. P004.01 (77828.0012)); U.S.patent application, filed concurrently herewith, entitled “MEDIA CONTENTCLIP IDENTIFICATION AND COMBINATION ARCHITECTURE” (Attorney Docket No.P006.01 (77828.0013)); and U.S. patent application, filed concurrentlyherewith, entitled “COMMENT DELIVERY AND FILTERING ARCHITECTURE”(Attorney Docket No. P007.01 (77828.0014)); each of which isincorporated herein in its entirety for all purposes.

BACKGROUND

The present invention relates to the creation, storage, and distributionof commentary on media content.

Most people interact quite regularly with recorded media content, suchas videos, music, books, and the spoken word. Modern society leansheavily on recorded media content to provide both entertainment andeducation. With the recent proliferation of portable media players,smartphones, tablet computers and the like, the demand for recordedmedia content continues to increase.

For many people, social interaction enhances the experience of viewingor listening to recorded media content. For example, the reactions andopinions of one's peers with respect to a certain film may increase thatperson's enjoyment of the film. In other examples, the perspective of afilmmaker or critic with a unique understanding of a particular filmmay, when shared, add greater meaning to one's perception of the film.

Often, a person consuming recorded media content may wish to sharecomments with other consumers of the recorded media content at presentor in the future. Additionally, it may be desirable to incorporatecomments generated by others with regard to specific portions ofrecorded media content into the experience of viewing and/or listeningto the media content.

SUMMARY

Methods, systems, and devices are described for associating commentswith playback of media content. In these methods, systems and devices, auser input device receives a selection of a first point during theplayback of the media content for insertion of a first comment. Theinput device may associate a first time code with the selected firstpoint. The input device may also receive a comment from the user at asecond point during the playback of the media content, the second pointbeing different from the first point. The comment is associated with thefirst time code.

In a first set of embodiments, a method of associating comments withplayback of media content includes receiving from a user at an inputdevice a selection of a first point during the playback of the mediacontent for insertion of a first comment. The method further includesdetermining a first time code associated with the selected first pointat the input device and receiving at the input device the first commentfrom the user at a second point during the playback of the mediacontent, the second point being different from the first point. Thefirst comment is then associated with the first time code.

In a second set of embodiments, an apparatus for associating commentswith playback of media content includes a selection module, a time codedetermination module, a comment receiving module, and an associationmodule. The selection receiving module is configured to receive from auser a selection of a first point during the playback of the mediacontent for insertion of a first comment. The time code determinationmodule is configured to determine a first time code associated with theselected first point. The comment receiving module is configured toreceive the first comment from the user at a second point during theplayback of the media content, the second point being different from thefirst point. The association module is configured to associate the firstcomment with the first time code.

In a third set of embodiments, a system for associating comments withplayback of media content includes an input device and a data store. Theinput device is configured to receive from a user a selection of a firstpoint during the playback of the media content for insertion of a firstcomment; determine a first time code associated with the selected firstpoint; receive the first comment from the user at a second point duringthe playback of the media content, the second point being different fromthe first point; and associate the first comment with the first timecode. The data store is in communication with the input device andassociated with the media content. The data store is configured to:receive the first comment and the first time code from the input deviceand store an association between the first comment and the first timecode.

In a fourth set of embodiments, a method of associating supplementalcomments with playback of media content includes receiving at an inputdevice a first comment and a first point during the playback of themedia content, the first comment associated with the first point. Thefirst comment and the first point during the playback of the mediacontent are received at a second point during the playback of the mediacontent, the second point being different from the first point.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the nature and advantages of the presentinvention may be realized by reference to the following drawings. In theappended figures, similar components or features may have the samereference label. Further, various components of the same type may bedistinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

FIG. 1 is a block diagram of an example system including componentsconfigured according to various embodiments of the invention.

FIG. 2 is a block diagram of an example display with an interface formaking comments according to various embodiments of the invention.

FIG. 3A is a block diagram of an example display with an interface formaking comments according to various embodiments of the invention.

FIG. 3B is a block diagram of an example display with an interface formaking comments according to various embodiments of the invention.

FIG. 3C is a block diagram of an example display with an interface formaking comments according to various embodiments of the invention.

FIG. 4 is a block diagram of an example system including componentsconfigured according to various embodiments of the invention.

FIG. 5 is a block diagram of an example system including componentsconfigured according to various embodiments of the invention.

FIG. 6 is a block diagram of an example system including componentsconfigured according to various embodiments of the invention.

FIG. 7 is a block diagram of an example user input device according tovarious embodiments of the invention.

FIG. 8 is a block diagram of an example system including componentsconfigured according to various embodiments of the invention.

FIG. 9 is a flowchart diagram of an example method of associatingcomments with playback of media content according to various embodimentsof the invention.

FIG. 10 is a flowchart diagram of an example method of associatingcomments with playback of media content according to various embodimentsof the invention.

FIG. 11 is a schematic diagram that illustrates a representative devicestructure that may be used in various embodiments of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

Methods, systems, devices, and computer program products are describedto create time-specific comments on multimedia and other informationcontent. A user interface allows a commentator to create a comment,associated with a time code or other locator, about the underlyingcontent (which, for purposes of this disclosure may include a movie, TVshow, Internet and other video, book, article, song or other audiorecording, photograph or other image, commercial advertisement, videogame, immersive media, augmented or artificial reality media, thecontent contained in a comment, or other displayed content). Thiscreated comment may be viewable (if marked public) to others watchingthe content in real time, or on a delayed basis. By using a modifiedtime code as an external reference, the comment need not actually beinserted into the underlying content.

This description provides examples only, and is not intended to limitthe scope, applicability, or configuration of the invention. Rather, theensuing description of the embodiments will provide those skilled in theart with an enabling description for implementing embodiments of theinvention. Various changes may be made in the function and arrangementof elements without departing from the spirit and scope of theinvention.

Thus, various embodiments may omit, substitute, or add variousprocedures or components as appropriate. For instance, it should beappreciated that in alternative embodiments the methods may be performedin an order different from that described, and that various steps may beadded, omitted, or combined. Also, features described with respect tocertain embodiments may be combined in various other embodiments.Different aspects and elements of the embodiments may be combined in asimilar manner.

It should also be appreciated that the following systems, methods,devices, and software may individually or collectively be components ofa larger system, wherein other procedures may take precedence over orotherwise modify their application. Also, a number of steps may berequired before, after, or concurrently with the following embodiments.

Systems, devices, methods, and software are described for the creationof commentary on multimedia and other information content. In one set ofembodiments, shown in FIG. 1, a system 100 includes input devices 105(e.g., mobile device 105-a, mobile phone 105-b, laptop 105-c, tablet105-d, computer 105-e, or other computing devices), networks 110,central server computer system 115, data store 120, and output devices125 (e.g., mobile device 125-a, mobile phone 125-b, laptop 125-c, tablet125-d, computer 125-e, or other computing devices). Each of thesecomponents may be in communication with each other, directly orindirectly.

Time, screen location, and object-specific comments may be created formultimedia and other information content. A user interface of an inputdevice 105 allows a commentator to generate a comment, associated with atime code, for example, relating to the content (which, for purposes ofthis disclosure, may include a movie, TV show, Internet and other video,book, article, song or other audio recording, photograph or other image,commercial advertisement, video game, immersive media, augmented orartificial reality media, the content contained in a comment, or otherdisplayed content). This created comment may be viewable to otherswatching the content in real time, or on a delayed basis. By using atime code, modified time code or other locator or combination oflocators as an external reference, the comment need not actually beinserted into the multimedia, but may be called up from remote serversas the user reaches the applicable point. The comment may include text,video, audio, photographs and other images, graphical overlays,animations, musical notations, geographic coordinates, discussionthreads, external and internal links and associations with media,meta-media or other comments, software applications and applets, specialnotations to set and grant permissions and define system behaviors orany combination thereof.

The comment may be stored locally (at the device 105, set-top box, orother storage device), or may be transmitted to the central servercomputer system 115 for cataloging and storage in data store 120. Thecentral server computer system 115 may be made up of one or more servercomputers, workstations, web servers, or other suitable computingdevices. The central server computer system 115 may be a cable orsatellite headend. The central server computer system 115 may be fullylocated within a single facility or distributed geographically, in whichcase a network may be used to integrate different components.

Data store 120 may be a single database, or may be made up of any numberof separate and distinct databases. The data store 120 may include one,or more, relational databases or components of relational databases(e.g., tables), object databases, or components of object databases,spreadsheets, text files, internal software lists, or any other type ofdata structure suitable for storing data. Thus, it should be appreciatedthat a data store 120 may each be multiple data storages (of the same ordifferent type), or may share a common data storage with other datastores. Although in some embodiments the data store 120 may be distinctfrom a central server computer system 115, in other embodiments it maybe integrated therein to varying degrees. The created commentary may beintegrated into the underlying multimedia or other information content,or may be stand-alone content to be leveraged with technology allowingthe time stamps to sync with the content as it is played.

The user may be alerted to the existence of a comment during playback ofthe content by the appearance of a viewing window or an icon that can betouched or clicked on output device 125, to reveal its contents. Inother examples, the content and commentary may be separated. Users maytailor their experience by selecting certain commentators, or types ofcommentators and/or comments.

The components of the system 100 may be directly connected, or may beconnected via a network 110 which may be any combination of thefollowing: the Internet, an IP network, an intranet, a wide-area network(“WAN”), a local-area network (“LAN”), a virtual private network, thePublic Switched Telephone Network (“PSTN”), or any other type of networksupporting data communication between devices described herein, indifferent embodiments. A network may include both wired and wirelessconnections, including optical links. Many other examples are possibleand apparent to those skilled in the art in light of this disclosure. Inthe discussion herein, a network may or may not be noted specifically.If no specific means of connection is noted, it may be assumed that thelink, communication, or other connection between devices may be via anetwork.

The following description describes various aspects and options for thesystem. In the discussion below, the comments may be referred to ascomments. Comments, as used herein, are to be interpreted very broadly.Comments may be created by users using input devices 105. Comments maybe viewed and controlled by users using output devices 125. An inputdevice 105 may be an output device 125, as well. Comments may be storedand organized in data store 120. Each feature is set forth for purposesof example only, and may be included or excluded in various embodiments.

A variety of interfaces may be used for comment creation. A commentatormay use these interfaces to input a comment into a movie or TV show (orother media) and to have that comment viewable (if the comment is markedpublic) to anyone else viewing the media content. By using a modifiedtime code as an external reference, the comment may not actually beinserted into the media or into any metadata, but may be called up froma remote server as a viewer reaches that point in the media. There maybe an uninterrupted feed by anticipating and pre-loading any upcomingcomments. The user may be alerted to the existence of a comment duringmedia playback by the appearance of a comment icon that can be touchedor clicked to reveal its contents.

Thus, in one embodiment, there may simply be a time marker, screenlocation, and associated text for a given piece of content. This may bestored separately and independently from the underlying content. FIG. 2illustrates an example block diagram 200 of a display 205 with aninterface for making comments. The display 205 may be on, or integratedwith, a mobile device, mobile phone, laptop, tablet, computer,television, head-mounted display, remote control, or any number of othercomputing or viewing devices. The display 205 may be a touch screen orbe controlled by user inputs into a remote, peripheral, or other mobiledevice. In the illustrated display 205, the locate comment icon 210 isoverlaid on the content. However, in other embodiments, the interfacemay be independent or on another screen or window from the content.

Display 205-a illustrates the interface at Time1, with a comment icon210. A user may click on, or otherwise select, the comment icon 210.Display 205-b illustrates the interface at Time2. A comment entry window215 appears in response to the selection of the comment icon to allowthe user to comment via text entry.

A comment may be set to display for a preset, or configurable, period. Auser may set in and out points so that an entire scene can be referencedby the comment. This may allow the user to ‘capture’ the scene or mediasub-unit completely for reference and for playback or sharing. The usermay set the comment to display at the beginning and end point, and to,optionally, display a scene marker during the span of the scene. Thisscene marker may be distinguishable from the standard comment marker. Auser or the system also may set in and out points in combination withthe identification of an entity or object on screen to capture theentire scene in which that object or entity appears.

As noted, comments are not limited to text, but may include text, video,audio, photographs and other images, graphical overlays, animations,musical notations, geographic coordinates, discussion threads, externaland internal links and associations with media, meta-media or othercomments, software applications and applets, special notations to setand grant permissions and define system behaviors or any combinationthereof. Comments may be made by speaking, and the comment may bereplayed during play of the underlying content. Alternatively, there maybe speech to text conversion for making comments and text-to-speechconversion for listening to comments. A user may insert a video ofhimself or herself with speech commentary, or other video may beinserted. Comments may be placed in specific locations on the display205.

Comments from a number of users may be linked, associated, or otherwiseintegrated to create a comment set. A user may insert video, graphical,or audio overlays designed to be used simultaneously with media contentrather than in separate or separable windows so that the media isaugmented or fully joined by the message content. Comments may containsoftware applications that allow or trigger various actions within localor remote devices, software systems, or devices and systems within andoutside the control of the user or the company or entity deliveringthese services. In this way, a comment can control local devices, forexample, to reduce volume in one viewing window or pause or slow themedia, or provide other services to the user or other entities. Commentsmay be used, for example, to control or administer micropayment systemsfor media or premium commentary. Comments may contain a hyperlink topoints outside the comment systems and they also may contain a link toor an association with a comment within the comment system or they maycontain several potential links offering user a choice of links tomedia, media subunits and other comments in the form of a question,quiz, survey, or other device such as a software player that can linkthe comments and play the resulting media and meta-media. Comments andlink-containing comments may be daisy-chained or linked in ahub-and-spoke or other such arrangement to provide unique ways for usersto move through media or through portions of media.

Comments may be inserted in traditionally non-time-based media such aspaintings, photographs, and architectural renderings in such as way asto create simulations or animations that allow this media to take on thedimension of time. A commentary sequence might mirror the layering ofpaint on a canvas, the movement of a focal plane in a photograph fromthe farthest to the nearest point, or the construction sequence of abuilding. Such a commentary time flow simulation may then be used toinvite further commentary along this artificial time line. Similarartificial time lines may also be created within time-based media toalter the time flow of the media or to allow the media to containmultiple time line spurs and other forms. For example, the image on asingle video frame may be commented into multiple successive layers thatrepresent an incremental progression in time to show the construction ofthe movie set shown in the video frame.

Comments may be inserted as header comments designed to appear tosubsequent users in a viewing window associated with a point at the verybeginning of the media and to contain general commentary about a pieceof media not tied to a particular point on a timeline. Comments maycontain messaging devices to allow the user, automatically or not, tosend a message directly to another user or entity. Comments may includea device to exclude any user response from the public commentary ormessage stream so that it is viewable only as a private comment ormessage. Comments may have special status and functionality as supercomments that allow individuals and entities, for example, a wikiservice, to capture and distill various comments in a single comment oran ordered collection of comments, which can then be further vetted andadded to. These super comments also may be filterable into collectionsof commentary that match a user's interests and sensibilities. Variouscomment types and combinations may be clearly marked with symbols,colors, or other methods to alert users to their capabilities andlimitations.

In some embodiments, the windows position may be set automatically, ormay be adjusted or otherwise customized at a user's discretion.Additional tabs and controls may be added to allow a user to selectadditional features described elsewhere herein.

FIG. 3A illustrates an example block diagram 300A of a display 305 withan alternative interface for making comments. This display 305 may be anexample of the display 205 described with reference to FIG. 2. Thedisplay 305 may be on, or integrated with, a mobile device, mobilephone, laptop, tablet, computer, television, head-mounted display,remote control, or any number of other computing or viewing devices. Thedisplay 305 may be a touch screen or be controlled by user inputs into aremote, peripheral, or other mobile device. In the illustrated display305, an insert icon 315 and scrubber control 310 are overlaid on thecontent. However, in other embodiments, the interface may be independentor on another screen or window from the content. The scrubber control310 allows a user to control the insertion timing within the content(e.g., within a 10 minute window).

Display 305-a illustrates the interface at Time1, with an insert icon315 and scrubber control 310 overlaid on the content. A user may use thescrubber control 310 to identify the proper place for comment insertion,and then may click on, or otherwise select, the insert icon 315. Display305-b illustrates the interface at Time2. A comment type window 320appears in response to the selection of the insert icon 315 to allow theuser to comment via text, audio, video, or hyperlink, or insert a pause,or adjust the insertion point.

FIG. 3B is a block diagram 325 of a display 330 with an alternativeinterface for making comments. This display 330 may be an example of thedisplay 205 described with reference to FIG. 2. The display 330 may beon, or integrated with, a mobile device, mobile phone, laptop, tablet,computer, television, head-mounted display, remote control, or anynumber of other computing or viewing devices. The display 330 may be atouch screen or be controlled by user inputs into a remote, peripheral,or other mobile device. In the illustrated display 330, an insert icon315-a, scrubber control 310-a, content window 335, comment stream fromother commentators 340, and comment entry window 345 are in differentwindows within the display 330. However, in other embodiments, eachinterface may be overlaid or integrated or on a screen, or some subsetof this functionality may be on a different device.

A user may use the scrubber control 310-a to identify the proper placefor comment insertion, and then may click on, or otherwise select, theinsert icon 315-a. This display configuration allows a user to view theunderlying content via content window 335, and also look at the commentstream 340 (perhaps filtered), and input comments into the comment entrywindow 345. In some examples, comments from the comment stream 340 maybe dragged and dropped into the comment entry window 345, or otherwiseselected for a comment set or other use. There may be threadeddiscussions, or comments from others may be pulled in.

A user may scrub a pane of commentary forward or back in its timelineand similarly scrub a media pane using software controls such as iconsor touch gestures. Users may be able to control whether commentary andmedia panes are coupled or decoupled and may use coupling and decouplingto control multiple panes independently or together. For example, a usermay be able to scrub both the comment stream window 340 and the contentwindow 335 by swiping in either pane when an interface is set to asynchronization state, such as a ‘hard sync’ state.

A user may be able to set or decouple a ‘hard sync’ with touch gesturessuch as swiping both the commentary stream window 340 and the contentwindow 335 simultaneously in the same direction to set a hard sync stateor by swiping both windows 340, 335 simultaneously in oppositedirections to turn off hard sync. Hard sync may be used to browse tomedia content at a desired location and view the relevant commentary,and/or to browse the commentary to view relevant moments in media. Incertain examples, a hard sync control may be resident in one window sothat swiping one window controls both windows while swiping the otherwindow only controls the window being manipulated, allowing a user tofreely manipulate one window out of synchronization without interruptingthe time progression on another window. Such controls also may be usedto manipulate multiple windows, and the set of windows being manipulatedmay or may not include a media window but may contain only commentary,for example. Other windows, such as the comment entry window 345 alsomay be set to control or be controlled by similar actions. Hard synccontrols may be automatically applied within the user environment or maybe set by the user.

These illustrative controls may allow a user to manage multiple windowseach with multiple time referents and then use further manipulation tomove or associate or perform other actions that allow a comment's timeassociations to be changed or to bridge multiple time associations.Doing so may allow a user to move a comment from one window to anotherto change its location in the media or to drop one comment onto anotherto associate the two comments. User actions to control synchronizationand decoupling may include other gestures and manipulations such astapping a handheld device or using swipe and other hand motions that donot contact a touchscreen but are otherwise sensed, or eye motionsdetected by the device and software or combinations of these gestures.

Resynchronization may be controlled by the user via software controlssuch as tapping in a window that represents the desired synchronizationpoint, or resynchronization may occur automatically when a usermanipulates a window so that it reaches the same point in the timelineas the other window. Automatic synchronization also may then beoverridden with continued manipulation. In certain examples whereautomatic synchronization is not available or is not desirable, a usermay be able to use commentary elements, such as a scene description or aline of dialogue to match the commentary with the underlying media, andspecial display configurations may be employed to facilitate thisfunctionality.

The system architecture for comment creation may take on a variety offorms. In some examples, DVD, broadcast, or streaming content may bereceived on a computer, set top box, or television. A user interfacecontrol (e.g., the control functions of the configurations illustratedin FIG. 2, 3A, or 3B) may be in the same screen, but distinct from thecontent (e.g., overlaid, or in separate windows). The user interfacecontrol and the content may be on different devices. The user interfacecontrol may use various mechanisms to associate a timestamp with theentry of a comment. In different embodiments, the user interface controlmay be integrated with content to varying degrees. The control windowmay receive inputs via a remote control, mobile device, keyboard, orother peripheral. The user interface control may control underlyingcontent (i.e., pausing underlying content), or not, and it also mayconvey the underlying content to other devices for viewing and otheractions.

FIG. 3C illustrates an example block diagram 350 of a display 355 withan alternative graphical overlay system for making, and viewing,comments from a single interface or set of interfaces. This display 355may be an example of the display 205 described with reference to FIG. 2.The display 355 may be on, or integrated with, a mobile device, mobilephone, laptop, tablet, computer, television, head-mounted display,remote control, or any number of other computing or viewing devices. Thedisplay 355 may be a touch screen or be controlled by user inputs into aremote, peripheral, or other mobile device.

Display 355-a illustrates the interface at Time1, with an array ofcircles 360 (which may be referred to as category beacons) overlaid onthe content. These beacons 360 may each represent a type of comment or acategory, or a comment source or source category and each beacon maylight up as the video passes a point at which a comment has beeninserted. If a beacon 360 is pressed and held, the comments filtered bycategory may be shown. The beacons 360 may show a steady glow forcomments that capture a media clip or subunit, with increasingbrightness when multiple commentators have chosen the subunit orportions thereof. Regular comments show as brief pulses. By tapping, orclicking on, any of these buttons the category may be displayed. Bypressing and holding, the user may be taken to a comment viewing window(not shown) that has been filtered by the category. The user cannavigate sideways to other comment windows in other categories, andvertically to comment windows filtered by type of comment, such ashumorous, or subject area, such as cinematography. Certain aspect ratios(not shown) may allow some of the components illustrated to be displayedbelow the video field by user preference. Letterbox video may be shiftedto the very top of the display to provide more room below. A window ofcomments may be displayed on one or more multiple lines, or incombination with other modular windows, such as a graphic to showtraffic in the comment stream.

Display 355-b illustrates the interface at Time2. A user may select theoption of having a graphical overlay appear, allowing the user to havecontrols to view comments, write a comment, scrub to comment insertionpoints, and other tasks. The graphical overlay may include a composecomment 365 set of features, and a current comment 370 set of features.

The compose comment 365 bar may be tapped, and the bar may expand to twolines and continue to expand as text is added. Below ‘send,’ on thesecond line is a pair of icons for audio and video recording of thecomment. ‘In’ may be tapped to insert the comment at the currentlocation in the media. It is not necessary to set an out point, but onecan be set by scrubbing to the desired location in the media andpressing the ‘out’ button. The ‘send’ feature provides sending optionssuch as ‘send as a Tweet,’ ‘send to Facebook,’ ‘send as a message,’ aswell as an option to ‘splice’ this comment to another in a commentdocument, using a splice or linking comment to connect them.

The current comment 370 bar shows the most recent comment passed in thetimeline. ‘Current comment’ may be tapped to view the entire comment.‘Current comment’ may be pressed and held to go to a comment viewingwindow. A small vertical line to the left of the current comment showswhen the current media has been captured within a comment clip, andgives a rough indication when more commentators also have chosen thecurrent scene. The triangles at either end of the bar can be tapped tomove forward and back through the comment timeline. ‘Add’ is a thumbsup, adding the comment to the user's set of comments that will be passedalong to others.

In this example, there is also a unified play, pause, and reverse button375. The button 375 may be designed to use a single control for mediaand other content windows. The default mode may be forward mode. A usermay tap once to play, tap again to pause, tap again to play, and so on.From a paused state, a user may press and hold for 1 second (or othertime period) to play in reverse (indicated, for example, by an audiblesound, 1×). A user may tap to pause, tap to play in reverse, tap topause. From a paused state in reverse mode, a user may press and holdfor 1 second (or other time period) to play in forward mode. Tappingpause and play continues to advance media forward. A user may press andhold again for reverse mode. There may also be fast forward and reverse.In either forward or reverse mode, at play or reverse play speed, a usermay press and hold the play button (e.g., holding for 1 second for 2×speed, holding for 2 seconds for 4× speed, and tapping to pause)

There may also be a split scrubber 380 that lets the user scan the mediaand the comment stream simultaneously, and to have control of eachindividually. To insert or view comments, a user slides a pair oftriangle markers along a double timeline and finds a desired location.The top scrubber controls the media (when the media window is on top orto the left) and the bottom scrubber controls the comment stream belowor to the right. The wide space between the lines controls them bothsimultaneously. Scrubber marks may disappear during finger (or cursor)swipe manipulation between the lines.

As the triangles are moved, they may change color or shading. Thisallows the user to be aware of a change in their position on thetimeline when their movement would not otherwise be obvious. If thetriangles are synchronized, the triangles may change color together,remaining the same color. If the triangles are out of synch, they mayeach change to different colors. This may help make the user aware thatthe user is out of synch when the distances on the timeline would nototherwise make it obvious.

With the split scrubber 380, a user may scan media and commentsindependently and quickly return the media to the place where it waspaused. By tapping (or clicking) one or another of the relevant trianglemarkers, the user can choose to have the media resynchronized to thetimeline location of either the media or the comments window. The usermay also resynchronize by initiating play in the appropriate window.Aside from scanning and browsing, the split scrubber 380 may be used tochange the insertion point or in/out points of a comment. The user mayselect a comment in the message window that he or she previously wrote,and after adjusting the media in the other window to the appropriatepoint, click the in or out icon, or click and drag the comment onto themedia.

The split scrubber 380 also may be of use with other combinations ofviewing windows, such as media and comment document windows, or commentand comment document windows. The split timeline may also be augmentedwith a third or fourth timeline to control multiple windows. The splitscrubber 380 may also display the in and out points of the currentcomment if the comment pertains to a media clip (as opposed to a commentthat has a single insertion point). It also may contain a space, forexample, between the two lines, in which a finger swipe in eitherdirection may advance the media to fine tune a location. It may containpoints at either end that can be clicked or tapped to advance or reversethe media one frame at a time or advance or reverse the comment streamone comment at a time. These points may additionally be controlled bypressing and holding them to advance media in gradual acceleration up to4×, for example. Fine control of this acceleration may be handled by aseries of press/hold maneuvers, such as one press/hold for 1× speed andtwo press/holds for 2×. The split scrubber 380 also may be used toimprove the speed of certain user operations when the system is unableto keep up with both media and commentary streams and when decoupling isdesired.

A fine tune scrubber bar 385 may consist of a line within a window(horizontal or vertical) designed to focus finger swipe motions into aspecific zone or to be manipulated by mouse control. It may be usedwithout split scrubber 380 (e.g., it may be used in configurations inwhich use of a split scrubber 380 could lead to confusion or in whichthe split scrubber is too bulky). In some examples, a fine tune scrubberbar 385 may be used with each window with timeline-related elements,such as media and commentary, served by its own scrubber. Manipulationof one window may control both windows unless the synchronization ofthese windows is decoupled. Windows may be decoupled by pressing andholding one window while manipulating the scrubber of another. Thewindows can be resynchronized by initiating ‘play’ in either window,which synchronizes play to the chosen window. The disappearing scrubmarker described herein may also have the capability of splitting intotop and bottom halves so that this scrubber can be used as a lessobtrusive split scrubber.

A fine tune scrubber bar 385 may be swiped from right to left, forexample, to advance the media or commentary forward in the timeline.Rapid and repeated swiping may trigger the appearance of a markerindicating speed, such as 2× or 4×. It may also trigger the appearanceof a scrub marker that shows progress along a timeline. When thesemarkers appear, it may be a signal to the user to stop swiping and allowthe content to continue at that speed to a desired point, or manipulatethe markers to move a greater distance. A tap may again pause thecontent so that it can be fine-tuned by finger swipe. Pressing andholding a midpoint of the line may also trigger the display of the scrubmarker and allow it to be manipulated to move greater distances in thetimeline and to its beginning or end. This marker may disappear when theuser makes further fine tune swiping motions. Tapping either end of theline may allow the user to advance frame by frame or comment by commentdepending on the window. Pressing and holding one end of the line mayincrease scrubbing speed by successive increments, accelerating thespeed from 2× to 4×, for example. This may work whether or not the scrubmarker is visible, though it may also cause the scrub marker to appearso the user is aware of motion in the timeline. Releasing this may causethe window's content to pause again, or it may continue at that speeduntil the window is tapped to pause it. Further swiping may return theuser to fine adjustments.

This scrubber bar 385 also may show fine movement in the disappearingscrub marker(s) by changing the color of the marker and the markerhalves in the same manner as the split scrubber. These scrub markers mayalso be manipulated in the manner of the split scrubber above todecouple media from other windows and, by pressing and holding one orthe other, resynchronizing to the timeline of the desired window.

There may also be browse scrubber functionality. Browse mode may beactivated by pressing and holding a point in the media or commentwindow. Both may pause as soon as one is touched. Now either window maybe manipulated by horizontal finger swipes to scrub the media and thecomment stream forward and reverse. Rapid swiping in one direction(e.g., three times) may increase the speed of scrubbing and trigger thedisplay of an optional simple scrubber bar and marker that can bemanipulated to move greater distances in the timeline. If the optionalsimple scrubber bar is not manipulated, it may disappear after a shortinterval. If it is manipulated, once a finger is lifted from thescrubber, it may disappear and finger swiping may be automaticallyreactivated. While this swiping tool may default to simultaneous andsynchronized viewing of both windows, the windows may be decoupled bymanipulating one window with a swipe while pressing and holding theother. Once decoupled, the held window no longer needs to be held, andit stays in its position in the timeline.

In browse mode, there may be one screen anchored in a position in thetimeline if the user hopes to return both windows to this originallocation to continue a viewing session. When it is time toresynchronize, the user can press and hold the window that is currentlyat the desired location, and the windows are synched. This mirrors asimilar operation on the split scrubber in which the windows areresynchronized by pressing and holding the triangle marker associatedwith the window that is in the desired timeline position. If a usermistakenly presses the wrong window, the undo command may return thewindows to their previous positions. The interface also may contain acommand that can return the media to its previous viewing position(s) incase many user actions have transpired since the viewing session wasinterrupted. This command may be a second tier option of the undocommand.

Browse mode may also allow the user to manipulate the media and commentscreens vertically. Vertical strokes in the media window may movethrough other media in the queue or its sub-queues or through othercomment projects underway. Pressing and holding points at the top andbottom edges of the window may move through sub-queues and other ofthese categories. This vertical movement may be helpful when the user isworking across media to link scenes, create timeline-independent commentsets, build annotated music playlists, or create other commentary/mediadocuments. It also allows a user to switch gears and enjoy other media.

The comment window may continue to display commentary from the firstmedia until the user presses and holds the media window to synchcommentary to that media and location on the timeline. Vertical strokesin the comment window move the window through the current list ofcomments arranged in timeline order, and each comment is expandable witha tap. Pressing and holding points at the top and bottom edges of thewindow may move through categories and classes of commentary so the usercan view comments on a certain topic or of a certain type, such ashumorous. Vertical movement in the comment window may be used inconjunction with horizontal movement to find a category (verticalmovement to ‘acting’) and then move horizontally to select a type ofcomment, such as humorous, to display humorous comments about theacting. In windows or operations that benefit from diagonal movement,such as forms of a comment document builder, finger swipes can movescreen objects diagonally and may move through matrices diagonally aswell. Matrices and other mapping devices may also be viewed andnavigated via other constructs in a 3D or simulated 3D environment, andthe element of movement through time may add a fourth dimension to thisnavigation.

Browse mode may be further augmented with controls that allow the userto tap the edges of a viewing window to control incremental movements.In the media window, for example, a tap to the right edge (or aspecifically marked spot on the edge) may advance the media one frame.In the message window, a similar tap may advance the comment stream byone comment. Taps on the top and bottom edges may advance the mediawindow through the queue or sub-queues, and move the comment windowthrough the comment categories. Pressing and holding these edges ormarks may move the material in the window to its beginning or end point,such as the end of a media. Finger swiping may be augmented orsubstituted by comparable mousing controls, such as clicking anddragging within a control bar, or with eye movements, such as two rapidglances toward one edge of the screen when a specific viewing window isselected.

In one set of embodiments, shown in FIG. 4, a system 400 includes inputdevice 105-g (e.g., a computer or television), network 110-c, centralserver computer system 115-a, and data store 120-a. Although in thiscase a central server computer system 115-a, the underlying content maybe accessed from a local source (e.g., DVD or other locally storedmultimedia). This system 400 may be the system 100 of FIG. 1. Each ofthese components may be in communication with each other, directly orindirectly.

The central server computer system 115-a may stream, or otherwisetransmit, video data (or other information data) to the input device.This content may be accessed from data store 120-a.

Time, screen location, and object-specific comments may be created forthe content. A user interface control of an input device 105-g allows auser to generate a comment, associated with a time code, for example,relating to the content (which, for purposes of this disclosure, mayinclude a movie, TV show, Internet and other video, book, article, songor other audio recording, photograph or other image, commercialadvertisement, video game, immersive media, augmented or artificialreality media, the content contained in a comment, or other displayedcontent). By using a time code, modified time code, or other locator orcombination of locators as an external reference, the comment need notactually be inserted into the multimedia. The comment may include text,video, audio, photographs and other images, graphical overlays,animations, musical notations, geographic coordinates, discussionthreads, external and internal links and associations with media,meta-media or other comments, software applications and applets, specialnotations to set and grant permissions and define system behaviors orany combination thereof. The user interface control may be generatedlocally, or served from the central server computer system 115-a to theinput device 105-g via network 110-c.

In one example, the content stream and user interface control areindependent and distinct from each other (even when both are on the samedisplay). In other embodiments, the content stream and user interfacecontrol are overlaid or partially integrated. In still otherembodiments, the content stream and user interface control are tightlyintegrated. The following examples further illustrate the options.

FIG. 5 illustrates a system 500 which includes a content source 505(which may be a DVD or other video disc player, a local or remotestorage device, a central server computer system), a set top box 510, amobile device 105-h (e.g., a tablet, smartphone, remote control, cellphone, or laptop), and a display 530 (e.g., television or otherdisplay). The set top box 510 includes a distribution module 515,content processing module 520, and local wireless device interfacemodule 525. Each of these components may be in communication with eachother, directly or indirectly. This system 500 may be the system 100,400 of FIG. 1 or 4.

In one example, the content source 505 streams media content to the settop box 510, which is processed by the content processing module 520.The content processing module 520 may create an echo version of themedia content. As used herein, the terms “parallel echo version” or“echo version” refer to a representation of the media content displayedconcurrently with a playback of a standard version of the media content.In certain examples, the “echo version” may simply be a lower resolutionor lower bandwidth version of the media content. Additionally oralternatively, the echo version may modify or remove certain aspects ofthe media content, such as replacing visual objects in the media contentwith outlines or removing a certain range of frequencies from an audiocomponent of the media content.

In parallel (or substantially in parallel), the content processingmodule 520 may provide a regular or high definition version of thecontent to distribution module 515 for delivery to display 530, whilealso delivering the echo version to the local wireless device interfacemodule 525. Local wireless device interface module 525 may deliver theecho version wirelessly to the mobile device 105-h. The mobile device105-h may control the playback of the echo to time the insertion ofcomments from the mobile device 105-h, and this echo timing may bedistinct and independent from the content playback on the display 530(i.e., the echo version and regular/high definition are notsynchronized). In other embodiments, the playback of the echo versionand the regular/high definition version are synchronized (so thatcontrol to stop, pause, fast forward, rewind at the display 530 (for theregular/high definition version) or the mobile device 105-h (for theecho version or the regular/high definition version) controls bothdisplays).

Local wireless device interface module 525 may generate a user interfacecontrol for delivery to the mobile device 105-h. This user interfacecontrol may be transmitted with the echo version of the content tomobile device 105-h. Alternatively, the user interface control may begenerated locally (e.g., as an App stored at the mobile device 105-h,wherein the echo version is integrated with the user interface controlat the mobile device 105-h.

In certain examples, the functionality of the set top box 510 shown inFIG. 5 may be performed by an alternate device, such as the contentsource 505, a special-purpose internet appliance, a server, a networkrouter, a network switch, and/or or a network access point.

In another example, the content source 505 streams different versions(the echo version and the regular/high definition version) of thecontent to the set top box 510, each of which may be stored by thecontent processing module 520. The content processing module 520 mayprovide the regular or high definition version of the content todistribution module 515 for delivery to display 530, while alsodelivering the echo version to the local wireless device interfacemodule 525. Local wireless device interface module 525 may deliver theecho version wirelessly to the mobile device 105-h. The mobile device105-h may control the playback of the echo version to time the insertionof comments, and this echo timing may be distinct and independent fromthe content playback on the display 530. In other embodiments, theplayback of the echo version and the regular/high definition version aresynchronized.

There are a number of other ways an echo version may be pulled out ofthe stream for time code needs. Echo version creation may occur at acentral server computer system, on the set top box 510, at the display530, or at the device 105-h. This echo version may also be designed sothat it satisfies the need for 1) time code synchronization, 2) roughvisual cross-checking to help users ensure that comments match up withdesired input location, 3) scrubbing for fine tuning of commentplacement, 4) delivery to comment servers to cross-check or identifymedia or provide synchronization, 5) faster retrieval, viewing, andselecting of media clips, or 6) any combination of these uses.

This echo version may be 1) a simple time code conveyance such as amedia time signal from one local device to another or from a mediadistribution device, 2) a low resolution version of the media contentthat stays within the bounds of copyright restrictions, 3) a version ofthe media content that is encoded to be inaccessible beyond time codeuse, 4) a version of the media content that is altered to berecognizable by the user but not in its standard and enjoyable form(such as music with a significant section of the frequency rangeremoved, or such as video with only rough object outlines), or 5) aversion of the media that is altered to be usable only by, for example,a data system designed to match audio waveforms, equivalent forms ofvideo, and equivalent forms of text and other multimedia, or 6) anycombination of these uses. An equivalent form of text, for example,might utilize word counts, word and letter count sequences, or a verydifficult to read font or combination of fonts.

FIG. 6 illustrates a system 600 which includes a content source 605(which may be a DVD or other video disc player, a local or remotestorage device, a central server computer system), a display 610 (e.g.,television, monitor, or other display), a mobile device 105-i (e.g., atablet, smartphone, remote control, cell phone, or laptop), and network110-d. Each of these components may be in communication with each other,directly or indirectly. This system 600 may be the system 100, 400 ofFIG. 1 or 4.

In one example, the content source 605 streams or otherwise transmitscontent to the display 610. The display 610 plays the content. Themobile device 105-i captures audio and/or video from the display (e.g.,through a microphone and camera). The mobile device 105-i may transmitthe captured audio or video over the network 110-d (e.g., to a centralserver computer system 115 of FIG. 1), and a time code associated withthe audio/video waveform may be generated to identify the content and/orthe location within the content. Alternatively, an App at the mobiledevice 105-i may perform some of these functions. A user may then entercomments about the content from the mobile device 115-i, and thecomments may be associated with the time code generated from theaudio/video waveform. A variety of audio and/or video recognitiontechniques known in the art may be used to facilitate contentrecognition.

One or more time code(s) for a comment may be generated by one useraction, and the comment may be created or edited and associated with themedia in a later action. Comments may be characterized or categorized orassociated in yet a later action by the original author or another user.The system may track the time and date of any additions, changes,characterizations, categorizations or associations to create a record ofthese changes for a range of purposes including the ability to track andanalyze the context in which a comment was made, the provenance anddevelopment of ideas discussed in commentary, the timing and sequence ofuser-made associations among comments, the timing of responses tocommercial and noncommercial comments, the timing of conversations andother interactions among users, and the progression and development ofcommentary-centric meta-media. Time and date may be used to allow usersto track and display their commentary history. Time and date may be usedto allow the creation of time-limited comments or to set comment aspectsand features that may change over time.

Comment time codes may be changed or adjusted by users after initialinsertion if needed. A user may reset the time code to another time. Auser may flag another user's comment for being incorrectly time codedand may suggest alternate time code placement. An entire set of commentsmay be shifted to new time codes if the initial registration of the timecodes was inaccurate. Time codes may be adjusted by the originalcommentator user by such means as changing the time code manually ormoving the comment into another media or comment display window that isassociated with the desired time code or adjustable to the desired timecode. Time codes also may be adjusted by moving the comment to a mappedtimeline or scrubber and locating the desired insertion point

In certain embodiments, users may be able to describe and categorizetheir comments and the comments of others using standardized terms suchas “academic”, “humorous”, “cinematography”, and “acting”. The usersalso may use personally relevant indexing terms to help them organizetheir comments and make them available to searches. A commentator mayhave different profiles (e.g., a number of nickname aliases associatedwith a username). This may allow the user to manage a set of onlinepersonalities for different friend groups and audiences and, forexample, to make only a subset of aliases available to commercialentities. All aliases may be transparently associated with a user. Usersmay create profile pages for defining an identity. Profile pages mayallow users to share recommendations, post personal montages, mashups,and best-of collections, post comment sets and favorite commenter lists,and receive and send messages. Thus, users may make comments underdifferent profiles, may limit who knows of and can access comments of orsend messages to each profile, may have private profiles, and may limitaccess by certain classes of users, such as commercial users. A user maylimit or select the users who can view comments, allowing a user tocomment privately to select friends or family members. The feature alsomay be used to allow limited distribution of comments by, say, an actoror director to users who belong to a special group or who pay a premium.There may be the option to time-limit a comment with an expirationfeature (e.g., useful for special-event commenting or for people whotend to do casual commenting with friends and do not wish their commentsto live on in perpetuity). Users may be characterized and categorized bythemselves, other users, and the system to segregate and integrate usersand user groupings, assist users in the discovery of compatible users,and further improve methods for filtering commentary.

As referred to generally above, a user may ‘package’ a set of comments.In one example, there may be a dedicated screen in which the user canwork to refine comments. A user may create comments but have them ‘onhold’ and undistributed so that user can assemble and edit a comment setand/or fine-tune it before making it public. A user may write commentsfor his or her eyes only, indefinitely. A user may apportion comments tovarious profiles. A user may “package” sets of comments less formally by“adding” or re-commenting the comments as the viewer encounters themduring a commenting session in which the user also may compose his orher own comments. A user may be able to make associations amongcomments, and these associations may be well defined and described andaccompanied by explanatory comments or they may be undefined. Becausethe system allows users the ability to view a single commentary track, asubsequent user can easily follow another user's self-generatedcommentary mingled with selected commentary by others.

A user may place non-location, and non-time, -specific comments at thefront or beginning of content, or in another such virtual space, toserve as overview and introduction of media by user. This may allow aviewer to pre-select commentators and comment sets for viewing and toprovide opportunities to make adjustments later. A viewer may click on acommentator and see that commentator's comments within the content,along with lists of other media that user has commented (with commentcounts and ratings).

In one example, these comments may be placed in a “media header” made upof commentary space that refers to the entire movie, TV episode, song,or other media item. The media header may contain multiple aspects,including an area for users to insert general commentary, orintroductions of their timeline commentary, or to collect notes andother material that may or may not be available to other users. Themedia header may contain, catalog and index material associated with themedia item or associated material, such as a movie database hosted byothers. The media header may offer tools to allow a user to associate orreference any non-timeline-associated material with points or units on amedia timeline, allowing, for example, a user to tag scene references ina review with the appropriate points in the media or link an actor'sbiography with scenes in which she appears.

The media header may further include an area for content owners ordistributors to insert official general commentary and information, andan area in which users and content owners and others may work togetherto fill in general information about the media, perhaps through a wikior other process. It may be used by content owners to set permissionsfor the use of the media. The media header may also contain a facilityfor making some of this information available to other services andentities or for users outside the comment software environment. Forexample, some of the information collected may be available to anyonewishing to correct or complete the metadata contained in their musicfiles.

The media header may similarly provide a central place for collectingand displaying dialogue, actor names, objects on screen or any otherinformation gleaned or indexed from time-based commentary. The mediaheader may contain applications or other facilities for analyzing dataor performing other functions. The media header may further contain mapsand other graphic representation of data, including associations amongcomments.

The media header also may display material that is nottimeline-associated, such as a synopsis or review or any othercommentary that refers to the media item as a whole. It may contain,catalog and index any material associated with the media item or it maysimilarly contain, catalog and index linkages to associated material,such as a movie database hosted by others. The media header may offerfeatures to allow a user to associate or reference anynon-timeline-associated material with points or units on a mediatimeline, allowing, for example, a user to tag scene references in areview with the appropriate points in the media or link an actor'sbiography with scenes in which she appears.

The media header space may be visible to a user when a media item iscalled up by the user and at the zero mark of its timeline. Additionallyor alternatively, the media header may be accessible to the user atother times, whether or not the media item itself is called up andavailable to the user. A media header comment can contain or embody anyform of media, and this media also may serve as a narrative or a hub orother means of organizing comments in ways that may depart from theoriginal timeline order. In certain examples, a header may be used as apublishing platform for media that references media and commentary andincorporates timeline-based linkages and associations

A user may create comment sets outside the time flow of a particularfilm or piece of media and across all media types and distributionplatforms. These narrative comment sets may be referred to as “commentdocuments” to distinguish them from comment sets based on the time flowof a single piece of original media.

Comment documents may allow a user to create a comment set that followsa particular idea or narrative outside of the media time flow and acrossmedia titles and types. A user workspace may also allow a user toconstruct a narrative from comment to comment, providing tools for theviewer to search for desired media and comments, insert them into theset, change the order of comments, and provide additional narration ornotation associated with individual comments within the flow or to serveas introductory material at the beginning of the comment set (and mediaplay). This narration may be delivered as a specialized comment, similarbut distinct from a comment that is a response to another comment (seethreaded comment, below).

A narrative or linking comment may be viewable within a window designedto allow a user to follow the comment-to-comment flow of a commentdocument. This window may allow the user to see the original commentthat the author of the comment document refers to, along with thenarrative comment (if there is one) that introduces or discusses thiscomment along with the media clip if the original comment contains an inand out point. Any appropriation of this comment document, or anyportion of it (down to a single link between two comments or clips), byanother user may contain this pairing of narrative and original commentsso that the user/author of that link between two comments may continueto have credit for making the association. This attribution may or maynot be visible to the user if the user elects not to see this level ofdetail, but it may be available to the user. A comment document may alsoserve as an easily annotated playlist tool that works across multiplemedia distribution platforms and multiple media types.

A comment document or portion thereof may be lifted and become part ofanother comment document and may be trimmed by a user. A user may trimthe comment document by removing comments on either side of the desiredsegment, but not by altering the order of the comments within thesegment, which would still contain the narrative comments associatedwith it. This, again, may allow the original author of the commentdocument to be credited with making this association, and yet it alsoallows other users to freely pass along or re-comment portions of thesedocuments and the associations they make among media clips.

A comment document may be as simple as a link between two clips withoutan accompanying narrative comment (e.g., a ‘comment document lite’interface that is popup based or otherwise very simple to use may allowreference or association linking one comment to another in the currentmedia file or elsewhere). A comment document may also be created byautomated systems that create linkages based on user associations amongcomments or similar associations inferred by the system.

A comment document may, therefore, be a collection of comments and mediaassembled in any order the user desires. It may be a collection ofscenes or music clips or other media sub-units. By using a speciallinking comment, the user may connect multiple comments in a daisychain, hub and spoke, or other configuration. Because a comment cancontain in and out points to define a content comment clip, a userlinking these together may create a guided tour that plays the mediaclips and describes where the tour is going and why. Informationaggregated from the clip selections by multiple users may be mapped orotherwise analyzed to show popularity, associative linkages and otherinformation that may have use in the creation and manipulation of mediasub-units by users and automated systems. Such analysis may be used, forexample, in automated systems to create meta-media or to micro-pricemedia sub-units. A user may also select clips to add to a personallibrary of clips from various sources for later use in meta-media or forother uses.

Commentary may be contained in either the media comment or the linkingcomment, depending on the result the user desires. If a user only wantsto introduce the clip associated with the media comment, or perhapsexplain why the user has chosen that clip, he or she may place thecommentary in the linking clip. If the user desires commentary to bedisplayed or played while the media plays, the user may add it to thecomment that relates to the media. In certain examples, the user maychoose both options.

In one example, a user may review a movie. The user may start with alinking comment and add the first couple of paragraphs. The user alsomay speak these paragraphs or make a video. As the user nears his or herfirst clip, the user may add a word or two of introduction and thencreate a regular comment that defines the in and out points of a scene.That comment also may contain commentary, and since the user may want tomake sure his or her audience does not have to read the commentary whileviewing the scene, the user may speak the comment and time its play to aprecise point on the timeline.

In certain examples, users may associate their comments with thecomments of others, and the system may make further associations amongcomments based on social connections, user interests, sentiment orlanguage analysis and/or other relevant factors. Secondary associationsmay be tracked across multiple degrees of separation. For example, anassociation between a comment made by a user and a comment made by afriend of that user in a different movie may be tracked. Someassociations may be weighted more or less than others. Theseassociations may be used to help users discover commentary and otherusers and media that may be of interest. Associations may be used tosuggest or deliver commentary and recommendations that a user mightotherwise not select intentionally. Associations may be used for trendanalysis, or for analysis of associative routes through commentary,media, users and groups. Associations may be mapped and displayed togive users access to associative information and to allow them toexplore these associations and better understand their meaning.

In certain examples, clusters of associated comments may be displayed in3D and manipulated with multitouch and other gestures to create, refineor change groupings that may represent categories of comments andassociations, the relative strength of associative connections, andother aspects. These and other manipulations may allow users to freelymove comments and groups of comments, and weaken or strengthenassociations, and these manipulations may be further assisted byautomated mapping and other representations.

The comment may contain special controls to manipulate the media as themedia is played. For example, the user may have the volume of a movedrop as the user's voice begins on the comment. The user may continue tocompose comments and linking comments ad infinitum. These options may bepossible with any media. In certain examples, the user may mix mediatypes, taking the audience of a film to the original scene in a novel,for instance, in either a written or audiobook form. A user may have theability to change or choose the primary media form in which commentaryand media are delivered and in which meta-media is constructed so that auser may, for example, have a passage from a book shown in a video orhave an open book handed to him or her in an artificial realityenvironment. A user also may make comment documents that allow theirviewers to choose their own path through the media the user hasselected, since linking comments may contain multiple options. A usermay give viewers the option of skipping a side tour. Alternatively, theuser may create a game or a quiz that leads an audience on multiplepaths.

As noted above, there may be an option for a threaded discussion, andsuch comments may be marked so that users are aware that the markedcomments may not be displayed as stand-alone comments. A user maydesignate a comment as both a stand-alone and threaded comment. A usermay alternately choose to associate a comment with another comment andhave that association serve as a response mechanism that allows thecomment to stand alone and yet also respond or illuminate the originalcomment.

Threaded discussions in the commentary may be handled so that the usermay navigate horizontally, vertically, or in combination to view mappedcomments even on a small screen. A user may choose to ‘add’ any commentwithin a discussion so that this comment may become part of the set ofcomments that is associated with this particular user and media. Theuser may in this way be able to view and ‘package’ selected commentsfrom a discussion so that these selected comments may be available toanyone viewing this user's comment set. Since the add button also is animplicit vote for the value of any comment, this action also allows thesystem to take note of this vote so that well-received comments that maynot have received any other rating can be highlighted in certain viewingsituations. Color-coding or other effects may be used in discussion viewto provide the user a sense of the characteristics of a particularcomment. Discussion view may also be used to handle queries andresponses.

A user may create a comment set or document from a threaded discussionthat follows a favored argument or favored examples put forward byparticipants in the discussion. A user may also highlight a favored pathin a threaded discussion by selecting the favored comments and “adding”or “re-commenting” them. This favored path may then be viewable tosubsequent users following the first user's commentary set “channel”.

Comments and responses in a discussion thread may be organized anddisplayed in traditional arrays which may show the comments in aparticular time order and address certain users' topics raised in boththe original comment and its responses. The comments and responses mayalso be arrayed, highlighted, color coded or otherwise indicated in waysthat treat associations among comments, and/or social associations witha current user or that user's interests and sensibilities, as factors ofgreater or lesser importance in the comment display. For example, a usermay see more comments within a discussion that have been composed byfriends or address a favorite topic, such as movie set design. A usermay be able to set preferences and tolerances for this functionality,and may be able to change these settings on the fly. The user mayfurther be able to use these capabilities to select particular commentsand responses for a curated set of comments and responses, or a curatedpath, that may then be passed along to other users or may be used toinform the system of noteworthy associations and preferred comments andresponses. Commentary channels may be curated or one-source commentarytracks, and may provide users with more controlled commentaryexperiences within the system. Specialized commentary channels canprovide utilities such as subtitles or audio dubbing or insertion ofdeleted scenes. Channels also may combine the comment sets and commentdocuments of multiple users, or may include comments from other sources.A user may navigate threaded comments by mousing over or clicking asingle comment to reveal an expanded view of related comments laterallyand vertically. A user may filter comments within the threaded commentviewing window to allow user to see comments most likely to be ofinterest and reduce overload. A user also may initiate a synchronizedviewing and commentary session with a small group of remote users usingmethods of buffering the media and synchronizing any pauses initiatedwithin the group. Comments can be viewed or heard as audio with orwithout pausing the media, though users may set automatic controls forpausing under certain circumstances.

As noted, there may be a number of different types of comments. Acomment may be a message unit (a packet of data). It may appear to havebeen inserted in a movie or other media. In some embodiments, it is notinserted in the media, but it is displayed, played, invoked or otherwiseactivated at the right moment as if it were. Clip comments are similarto plain comments, except the user has added an ‘out’ point so thecomment can refer to an entire scene in a movie, or a clip from a song,instead of just the comment's insertion point. Splice comments mayconnect one comment to another. Splice comments may allow a user toassemble guided tours of favorite scenes, create annotated musicplaylists, write reviews with media clips displayed when and how theuser desires. Splice comments and other specialized comment types may ormay not need to be embedded directly in the media with a specific timecode but may relate only to other comments and may to some extent shareor reflect the time codes of those associated comments flexibly.

Query comments allow a user to ask questions and have friends andstrangers help sort out the answers. Query comments may reside in themedia, but they may also be sent as messages to users who considerthemselves experts on certain media and genres. When experts respond,their answers may also reside in the media as a threaded conversationwith the query as the starting point. Message comments allow a user torespond to a comment or query via a direct message rather than anembedded comment (e.g., that a user does not wish to make public).Message query comments may not collect their responses within the mediaas discussion. Rather, all responses may be sent as messages. These maybe useful for commercial commenting.

Header comments may contain commentary that pertains to the movie, song,or other content as a whole. For example, header commentary mayintroduce the set of comments that was made as a commentator watched amovie, or it may provide general comments about the movie. Headercomments may contain indexes, catalogues of commentary, analytic tools,mapping and graphing tools and other features that may assist users inexploring and analyzing commentary and media and associations therein.From the perspective of a user, header comments may occupy the spacejust before the movie begins. Permissions comments in the header andelsewhere may allow content owners and other entities to set permissionsfor how content may be accessed and used. App comments in the header andelsewhere may contain mini-apps or triggers that perform a functionwithin the software. An app comment may, for example, reduce the volumein the media window so that the clip selected for a comment document isquiet as a voiceover is played. Other comment elements andcustomizations may include: 1) authorizations for special event orpremium comment streams, 2) direct notifications sent when certaincomments or comment streams have been selected or operated upon, 3)automatic messaging that allows certain comments to generate a messagewhen the user responds, like query comments but without the experts, 4)multiple response comments that give users options for where a splicewill take them, creating various possibilities for questionnaires,quizzes, interactive meta media, and 5) super comments that allow usersto consolidate the general commentary at a point in the timeline or in aspecific category and have it presented and vetted through a processsuch as a wiki.

Referring now to FIG. 7, a block diagram of an example of a user inputdevice 105-j is shown. The user input device 105-j may interact with auser during playback of media content to receive and store commentsabout specific portions of the media content. The user input device105-j may be an example of the user input device 105-j described abovewith respect to FIG. 1, 4, or 6. The user input device 105-j may providea user interface which allows the user at least to generate commentsabout media content and associate the comments with discrete pointsduring the playback of the media content.

The user input device 105-j of the present example includes a selectionreceiving module 705, a time code determination module 710, a commentreceiving module 715, and an association module 720. Each of thesecomponents may be in communication, directly or indirectly. Theselection receiving module 705 may interact with the user to receivefrom the user a selection of a first point during the playback of themedia content for insertion of a first comment. This first point may bethe insertion point described above with reference to FIG. 3A, 3B, or 3Cabove.

The selection of the first point may be received, using any of a varietyof means. In certain examples, the user may select the first pointduring the playback of the media content using a scrubber control (e.g.,scrubber 310 of FIG. 3A) or a split scrubber control (e.g., splitscrubber 380 of FIG. 3C). Additionally or alternatively, the user mayselect the first point during the playback of the media content bytapping a button or making a menu selection. The first point may beselected in real time during the playback of the media content and/orretroactively.

In certain examples, the selection receiving module 705 may receivemultiple selections from the user of multiple points during the playbackof the media content for association with a single comment. Forinstance, as described above, the user may select a first point duringthe playback of the media content for insertion of a comment and asecond point during the playback of the media content to indicate anexpiration or termination of the display of the comment. Additionally oralternatively, the user may select a first point during the playback ofthe media content for insertion of a comment and a duration of time forwhich the comment is to be displayed. In these examples, a comment maybe displayed for a longer or shorter period of time than the duration ofthe media content or media content clip with which the comment isassociated.

The time code determination module 710 may be configured to generate atime code for each selected point during the playback of the mediacontent received at the selection receiving module 705. The time codegenerated may be based upon a timing associated with the playback of themedia content. The generated time code may serve as a globallyunderstood reference for the selected point(s) to be associated with agiven comment. That is, a device or process other than the user inputdevice 105-j may be able to interpret the time codes generated by thetime code determination module 710 to identify each selected pointassociated with a comment.

The comment receiving module 715 may be configured to receive a commentfrom the user. The user may intend that the comment be associated withone or more of the points during the playback of the media contentreceived by the selection receiving module 705. The comment may includetext and/or other content. In certain examples, the comment may includerecorded audio, video, hyperlinks, animated content, and/or othercontent. As described previously, some comments may even includesoftware functionality, such as applications or applets that may beinvoked when the comments are viewed. For instance, a comment mayinclude executable code such that when the comment is viewed, the codeis executed to display an interactive activity to the viewer. Thisfunctionality also may be as simple as an instruction to display thecomment for a set period of time that may not correspond to the durationof the underlying media clip.

In additional or alternative examples, the comment(s) received at thecomment receiving module 715 may be associated with one or more existingcomments to allow a viewer of the comments to move through a daisy chainor other linked organization of comments, as described previously. Thus,when a comment is received at the comment receiving module 715, the userinput device 105-j may be further configured to receive a selection of asecond comment from the user for association with the first comment.

Once the selection of one or more points during the playback of themedia content have been received by the selection receiving module 705and the comment has been received by the comment receiving module 715,the association module 720 may be configured to associate the receivedcomment with the received one or more points during the playback of themedia content. This association may make it possible for a device orprocess receiving the comment to identify the most relevant portions ofthe media content to which the comment applies. Where applicable, theassociation module 720 may also associate the received comment with oneor more other comments. An association with another comment may allowthe explicit or primary time code of a comment to be of secondaryimportance when placed in a user's media time line and may cause it tobe overridden or adjusted in favor of the time code of the othercomment, or it may be associated with multiple points on a time line tocorrespond with multiple comments. Associating one comment with anothermay also be used as a way to correct the placement of a misplacedcomment or suggest a new placement.

The comment may be received at the comment receiving module 715 at adifferent point during the playback of the media content than the pointselected for insertion of the comment. For example, a user watching avideo may select point A for insertion of a comment, work on the commentwhile the video continues to play, and submit the comment to the commentreceiving module 715 at point B. However, because of the selectedpoint(s) during the playback of the media content received at theselection receiving module 705, the association module 720 may associatethe received content with the point(s) during the playback of the mediacontent for which the comment is most relevant or for which the commentis intended.

The association of the received comment with the selected one or morepoints during the playback of the media content may be accomplished in anumber of ways. In certain examples, the comment may be annotated withone or more pointers to the selected one or more points during theplayback of the media content. Additionally or alternatively, a recordof the received points during the playback of the media content may beannotated with a pointer to the comment. In still other examples, acentral database may include references to comments and points duringthe playback of the media content, such that a reference to the receivedcomment in the central database is associated with references to theselected points during the playback of the media content.

In certain embodiments, the user input device 105-j may be the samedevice used by the user to view the playback of the media content. Forexample, the user input device 105-j may be a laptop or tablet computerwhich simultaneously plays the media content to the user and allows theuser to generate comments for association with specific points in themedia content. Alternatively, the user input device 105-j may beseparate from a device used to playback the media content. For example,the user input device 105-j may be a notebook or tablet computer onwhich the user enters comments while watching content provided to atelevision by a set top box. In such examples, some degree ofsynchronization may occur between the user input device 105-j and thedevice on which playback occurs.

Referring next to FIG. 8, an example of a system 800 is shown. In thesystem 800 of FIG. 8, a user 815 utilizes a user input device 105-k togenerate comments about the playback of media content being viewed on anoutput device 125-f The system 800 of the present example includes theuser input device 105-k, the output device 125-f, a network 110-e, acomment server 115-b, a comment data store 120-b, a media content server115-c, and a media content data store 120-c. Each of these componentsmay be in communication, directly or indirectly.

The system 800 may be an example of the system 100 described above withrespect to FIG. 1, the system 400 described above with respect to FIG.4, the system 500 described above with respect to FIG. 5, and/or thesystem 600 described above with respect to FIG. 6. The user input device105-k may be an example of the user input device 105 described abovewith respect to FIG. 1, 4, 6, or 7. The output device 125-f may be anexample of the output device 125 described above with reference toFIG. 1. The network 110-e may be an example of the network 110 describedabove with reference to FIG. 1, 4, or 6. The comment server 115-b andthe media content server 115-c may be examples of the central servercomputer system 115 described above with reference to FIG. 1 or 4. Thecomment data store 120-b and the media content data store 120-c may beexamples of the data store 120 described above with respect to FIG. 1 orFIG. 4.

Similar to the user input device 105-j of FIG. 7, the user input device105-k of the present example includes a selection receiving module 705-afor receiving a selection of one or more points during the playback ofthe media content from the user 815, a time code determination module710-a for generating time codes with respect to the selected one or morepoints during the playback of the media content, a comment receivingmodule 715-a for receiving a comment from the user 815, and anassociation module 720-a for associating the received comment with thetime codes for the selected one or more points during the playback ofthe media content.

The user input device 105-k of the present example also includes ascrubber control module 805 and a data store transmission module 810.The scrubber control module 805 may generate a scrubber control fordisplay to the user 815, as described above with reference to FIG. 3Aand FIG. 3C. The scrubber control may facilitate the selection of theone or more points during the playback of the media content forassociation with the received comment. The data store transmissionmodule 810 may transmit the selected one or more points during theplayback of the media content with the received comment over the network110-c to the comment server 115-b for storage in the comment data store120-b. The comment may remain associated with the selected one or morepoints during the playback of the media content in the comment datastore 120-b.

In the present example, the user may view the playback of the mediacontent on the output device 125-f while concurrently using the userinput device 105-k to generate comments associated with specific pointsduring the playback of the media content. In certain examples, thefunctionality of the user input device 105-k and the output device 125-fmay be implemented by a single user device. The output device 125-f mayreceive the media content from the media content data store 120-c by wayof the media content server 115-c and the network 110-e. In otherexamples, the media content may be played at the output device 125-ffrom a local disk or other form of local memory. In certain examples,the output device 125-f may retrieve comments relative to the mediacontent from the comment data store 120-b via the comment server 115-band the network 110-e for display during playback of the media content.

As shown in FIG. 8, the comment data store 120-b and the media contentdata store 120-c may be separate and independent from each other. Thecomments stored by the comment data store 120-b which are intended fordisplay with the media content may be associated with the media contentin the data store 120-b. In certain examples, the comment data store120-b itself may be associated with one or more articles of mediacontent.

Referring next to FIG. 9, flowchart of an example method 900 ofassociating comments with playback of media content is shown. The method900 may be performed, for example, by the user input device 105described above with respect to FIG. 1, 4, 6, or 7, the central servercomputer system 115 described above with reference to FIG. 1 or 4,and/or by the set-top box 510 described above with respect to FIG. 5.

At block 905, a selection of a first point during the playback of mediacontent is received from a user at an input device for insertion of acomment. At block 910, a time code associated with the first point isdetermined at the input device. At block 915, the input device receivesthe comment from the user at a second point during the playback of themedia content, the second point being different from the first point. Atblock 920, the comment is associated with the time code.

Referring next to FIG. 10, flowchart of another example method 1000 ofassociating comments with playback of media content is shown. The method1000 may be performed, for example, by the user input device 105described above with respect to FIG. 1, 4, 6, or 7, the central servercomputer system 115 described above with reference to FIG. 1 or 4,and/or by the set-top box 510 described above with respect to FIG. 5.

At block 1005, a selection of a first point during the playback of mediacontent is received from a user at an input device for insertion of acomment. At block 1010, a first time code associated with the firstselected point during the playback of the media content is determined atthe input device. At block 1015, the input device receives the commentfrom the user at a second point during the playback of the mediacontent, the second point being different from the first point. At block1020, the input device receives a selection of a third point during theplayback of the media content for terminating a display of the comment.At block 1025, a second time code associated with the third point duringthe playback of the media content is determined. At block 1030, thecomment is associated with the first and second time codes. At block1035, the comment and the first and second time codes are transmitted toa data store of stored comments associated with the media content.

A device structure 1100 that may be used for one or more input devices105, the central server computer system 115, for one or more set-topboxes 510, for one or more output devices 125, or for other computingdevices described herein, is illustrated with the schematic diagram ofFIG. 11. This drawing broadly illustrates how individual system elementsof each of the aforementioned devices may be implemented, whether in aseparated or more integrated manner. Thus, any or all of the variouscomponents of one of the aforementioned devices may be combined in asingle unit or separately maintained and can further be distributed inmultiple groupings or physical units or across multiple locations. Theexample structure shown is made up of hardware elements that areelectrically coupled via bus 1105, including processor(s) 1110 (whichmay further comprise a DSP or special-purpose processor), storagedevice(s) 1115, input device(s) 1120, and output device(s) 1125. Thestorage device(s) 1115 may be a machine-readable storage media readerconnected to any machine-readable storage medium, the combinationcomprehensively representing remote, local, fixed, or removable storagedevices or storage media for temporarily or more permanently containingcomputer-readable information. The communications system(s) interface1145 may interface to a wired, wireless, or other type of interfacingconnection that permits data to be exchanged with other devices. Thecommunications system(s) interface 1145 may permit data to be exchangedwith a network.

The structure 1100 may also include additional software elements, shownas being currently located within working memory 1130, including anoperating system 1135 and other code 1140, such as programs orapplications designed to implement methods of the invention. It will beapparent to those skilled in the art that substantial variations may beused in accordance with specific requirements. For example, customizedhardware might also be used, or particular elements might be implementedin hardware, software (including portable software, such as applets), orboth.

It should be noted that the methods, systems, and devices discussedabove are intended merely to be examples. It must be stressed thatvarious embodiments may omit, substitute, or add various procedures orcomponents as appropriate. For instance, it should be appreciated that,in alternative embodiments, the methods may be performed in an orderdifferent from that described, and that various steps may be added,omitted, or combined. Also, features described with respect to certainembodiments may be combined in various other embodiments. Differentaspects and elements of the embodiments may be combined in a similarmanner. Also, it should be emphasized that technology evolves and, thus,many of the elements are examples and should not be interpreted to limitthe scope of the invention.

Specific details are given in the description to provide a thoroughunderstanding of the embodiments. However, it will be understood by oneof ordinary skill in the art that the embodiments may be practicedwithout these specific details. For example, well-known circuits,processes, algorithms, structures, and techniques have been shownwithout unnecessary detail in order to avoid obscuring the embodiments.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flow diagram or block diagram. Although each maydescribe the operations as a sequential process, many of the operationscan be performed in parallel or concurrently. In addition, the order ofthe operations may be rearranged. A process may have additional stepsnot included in the figure.

Moreover, as disclosed herein, the term “memory” or “memory unit” mayrepresent one or more devices for storing data, including read-onlymemory (ROM), random access memory (RAM), magnetic RAM, core memory,magnetic disk storage mediums, optical storage mediums, flash memorydevices, or other computer-readable mediums for storing information. Theterm “computer-readable medium” includes, but is not limited to,portable or fixed storage devices, optical storage devices, wirelesschannels, a sim card, other smart cards, and various other mediumscapable of storing, containing, or carrying instructions or data.

Furthermore, embodiments may be implemented by hardware, software,firmware, middleware, microcode, hardware description languages, or anycombination thereof. When implemented in software, firmware, middleware,or microcode, the program code or code segments to perform the necessarytasks may be stored in a computer-readable medium such as a storagemedium. Processors may perform the necessary tasks.

A group of items linked with the conjunction “and” should not be read asrequiring that each and every one of those items be present in thegrouping, but rather should be read as “and/or” unless expressly statedotherwise. Similarly, a group of items linked with the conjunction “or”should not be read as requiring mutual exclusivity among that group, butrather should also be read as “and/or” unless expressly statedotherwise.

Having described several embodiments, it will be recognized by those ofskill in the art that various modifications, alternative constructions,and equivalents may be used without departing from the spirit of theinvention. For example, the above elements may merely be a component ofa larger system, wherein other rules may take precedence over orotherwise modify the application of the invention. Also, a number ofsteps may be undertaken before, during, or after the above elements areconsidered. Accordingly, the above description should not be taken aslimiting the scope of the invention.

These components may, individually or collectively, be implemented withone or more Application Specific Integrated Circuits (ASICs) adapted toperform some or all of the applicable functions in hardware.Alternatively, the functions may be performed by one or more otherprocessing units (or cores), on one or more integrated circuits. Inother embodiments, other types of integrated circuits may be used (e.g.,Structured/Platform ASICs, Field Programmable Gate Arrays (FPGAs) andother Semi-Custom ICs), which may be programmed in any manner known inthe art. The functions of each unit may also be implemented, in whole orin part, with instructions embodied in a memory, formatted to beexecuted by one or more general or application-specific processors.

1. A method of associating comments with playback of media content, themethod comprising: receiving from a user at an input device a selectionof a first point during the playback of the media content for insertionof a first comment; determining a first time code associated with theselected first point at the input device; receiving at the input devicethe first comment from the user at a second point during the playback ofthe media content, the second point being different from the firstpoint; and associating the first comment with the first time code. 2.The method of claim 1, further comprising: transmitting the firstcomment and the first time code to a data store of stored commentsassociated with the media content.
 3. The method of claim 2, wherein thedata store is stored independently from the media content.
 4. The methodof claim 1, further comprising: receiving a selection of a secondcomment from the user for association with the first comment; andassociating the first comment with the second comment.
 5. The method ofclaim 1, further comprising: receiving a selection of a third pointduring the playback of the media content for terminating a display ofthe first comment; determining a second time code associated with thethird point during the playback of the media content; and associatingthe first comment with the second time code.
 6. The method of claim 1,further comprising: receiving a selection of a software function fromthe user for association with the first comment; and associating thefirst comment with the selected software function.
 7. The method ofclaim 1, further comprising: displaying a scrubber control associatedwith comment insertion to the user; receiving a comment insertionindication from the user; and receiving the selection of the first pointby determining a position of the scrubber control when the commentinsertion indication is received.
 8. An apparatus for associatingcomments with playback of media content, the apparatus comprising: aselection receiving module configured to receive from a user a selectionof a first point during the playback of the media content for insertionof a first comment; a time code determination module configured todetermine a first time code associated with the selected first point; acomment receiving module configured to receive the first comment fromthe user at a second point during the playback of the media content, thesecond point being different from the first point; and an associationmodule configured to associate the first comment with the first timecode.
 9. The apparatus of claim 8, further comprising: a data storetransmission module configured to transmit the first comment and thefirst time code to a data store of stored comments associated with themedia content.
 10. The apparatus of claim 9, wherein the data store isstored independently from the media content.
 11. The apparatus of claim8, wherein: the selection receiving module is further configured toreceive a selection of a second comment from the user for associationwith the first comment; and the association module is further configuredto associate the first comment with the second comment.
 12. Theapparatus of claim 8, wherein: the selection receiving module is furtherconfigured to receive a selection of a third point during the playbackof the media content for terminating a display of the first comment; thetime code determination module is further configured to determine asecond time code associated with the third point during the playback ofthe media content; and the association module is further configured toassociate the first comment with the second time code.
 13. The apparatusof claim 8, further wherein: the selection receiving module is furtherconfigured to receive a selection of a software function from the userfor association with the first comment; and the association module isfurther configured to associate the first comment with the selectedsoftware function.
 14. The apparatus of claim 8, further comprising: ascrubber control module configured to display a scrubber controlassociated with comment insertion to the user; wherein the selectionreceiving module is further configured to receive the selection of thefirst point by: receiving a comment insertion indication from the user;and determining a position of the scrubber control when the commentinsertion indication is received.
 15. A system for associating commentswith playback of media content, the system comprising: an input deviceconfigured to: receive from a user a selection of a first point duringthe playback of the media content for insertion of a first comment;determine a first time code associated with the selected first point;receive the first comment from the user at a second point during theplayback of the media content, the second point being different from thefirst point; and associate the first comment with the first time code;and a data store in communication with the input device and associatedwith the media content, the data store configured to: receive the firstcomment and the first time code from the input device; and store anassociation between the first comment and the first time code.
 16. Thesystem of claim 15, wherein the data store is stored independently fromthe media content.
 17. The system of claim 15, wherein the input deviceis further configured to: receive a selection of a second comment fromthe user for association with the first comment; and associate the firstcomment with the second comment.
 18. The system of claim 15, wherein theinput device is further configured to: receive a selection of a thirdpoint during the playback of the media content for terminating a displayof the first comment; determine a second time code associated with thethird point during the playback of the media content; and associate thefirst comment with the second time code.
 19. The system of claim 15,wherein the input device is further configured to: display a scrubbercontrol associated with comment insertion to the user; receive a commentinsertion indication from the user; and receive the selection of thefirst point by determining a position of the scrubber control when thecomment insertion indication is received.
 20. A method of associatingsupplemental comments with playback of media content, the methodcomprising: receiving at an input device a first comment and a firstpoint during the playback of the media content, the first commentassociated with the first point; wherein the first comment and the firstpoint during the playback of the media content are received at a secondpoint during the playback of the media content, the second point beingdifferent from the first point.